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ABSTRACT 



This paper describes how student reactions shaped the design 
of a WWW (World Wide Web) -based course environment at the University of 
Twente (Netherlands), using the example of a first year course. The first 
section discusses the importance of user interface (re) design for WWW-based 
courses. The University's focus on student evaluations of WWW-based course 
environments is explained in the second section, specifically the student 
evaluations of the user interface of the 1996-97 ISM-1 (Instrumentation 
Technology 1) course. Approximately 85 students were asked on four separate 
occasions to respond to a course-site evaluation form, offered via a CGI 
(Common Gateway Interface) form. The next section covers the redesign of the 
user interface for the 1997-98 course environment, including navigation, 
layout, icons, tables, text clarity, and text readability. The student 
evaluation of the redesign, in which 38 out of 60 students responded to an 
optional survey during the fifth week of class, is described in the fourth 
section. The last section presents conclusions. Two figures illustrate the 
interface design of the ISM-1 course site from 1996-97 and 1997-98. A table 
presents a sample of items from student evaluation of the redesigned user 
interface of the 1997-98 course site. Contains 12 references. (DLS ) 
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Abstract: The user interface of a WWW-based (World Wide Web) course environment 
influences the users' mental characterization of his/her goals with the system. Although 
research has accumulated with respect to the user interface of computer-related products, 
little direct focus has developed for the new type of product: a WWW-based course 
environment. At the University of Twente we have been designing and using WWW-based 
course environments since 1994 and we systematically collect student reactions to the 
interfaces of those environments. In this report we describe how the student reactions shape 
our design of a WWW-based environment through the example of a first-year course. Most of 
the user interface redesign choices to the 1997-98 course were well received by the students. 
The major problem of site structure and how this is presented to the students via the user 
interface remains. 



1. The Importance of User Interface (Re)Design for WWW-Based Courses 

The user interface, the point of interaction between a user and a tool [Cox & Walker 1993], of a 
computer system has long been studied. Long lists of guidelines for user interface design have appeared, 
particularly within HCI (Human-Computer Interaction) research [Shneiderman 1987]. The user interface for 
educational software, including multimedia products, has also received regular attention in literature [Boyle 
1997]. 

The term ‘user interface’ does not seem to be much used with WWW environments yet, although 
many sets of guidelines have appeared for WWW systems that relate to the classic focuses in user interface 
design. The function of the user interface is helping the user to have an internal mental characterization of his 
or her goals with a system, helping the user select an action, helping the user execute the action, and helping 
the user evaluate the results of his or her interaction [Norman 1984]. 

WWW-based course environments have begun to receive attention in terms of design issues, but 
mostly in terms of ‘presentation design’ and ‘navigation support’ instead of user interface design. Eekma and 
Collis [Eekma & Collis 1996] have made a summary of 20 of those design guidelines for WWW-based course 
environments. Guidelines, when used, are often developed by theory and put into practice in a WWW-based 
course under development. Little evidence is found in literature about student evaluations regarding the 
interface design of WWW-based course environments. There are few evaluations present in which they are 
using the interface themselves and even less evidence of how such student evaluations have been systematically 
used as feedback for the redesign of WWW-based course environments over several cycles of a course. 
Undoubtedly this is mostly a result of the newness of the medium. At the Faculty of Educational Science and 
Technology at the University of Twente, WWW-based course environments have been in use since 1994 



ERjt 



p.1^7 AWA! 



’_v. \ r 



2 



[Collis 1997]. Enough time has passed and experience has developed so that a number of cycles of student 
evaluations of WWW-based course environments have occurred, each resulting in a substantial redesign of the 
environments based on these student evaluations. In this paper we report on one such recent cycle. 

In this report we describe how the student reactions shaped our design of a WWW-based environment 
through the example of a first-year course, and the redesign of its user interface between the 1996-97 and 
1997-98 versions, as well as our first student reactions to the 1997-98 redesign. 



2. Student Evaluations of the User Interface of WWW-Based Courses at the University 
of Twente 

The Faculty of Educational Science and Technology has a pioneering history of WWW-based course 
environments. This has facilitated the steady collection of student-evaluation data about these courses. Also 
important is the nature of the courses themselves that are offered through WWW-based environments. 
Primarily these are courses in which students are studying instructional design, the design of educational 
media or user interface design as content areas, which means that the students are particularly sensitive to the 
direct impact of design and have specific criteria and vocabulary for articulating their evaluative comments. 
Even first-year students, in the ISM-1 (Instrumentation Technology 1) course, not only evaluate the user 
interface design of WWW environments made by others, but also learn to evaluate the user interfaces of sites 
they design and make themselves. It is natural that regular opportunities for the students to evaluate the user 
interface of their own course-site are built into the course activities as the course makes extensive use of a 
WWW-based environment [Collis, Verhagen & Gervedink Nijhuis 1997a, b]. 

2.1 Student Evaluations of the User Interface of the 1996-97 ISM-1 Course 

An example of such evaluation by the first-year students occurred during the 1996-97 cycle of the 
ISM-1 course [Collis, Verhagen & Gervedink Nijhuis 1996a], a course that runs for an entire academic year. 
Approximately 85 students in the course were asked on four separate occasions to respond to a course-site 
evaluation form, offered via a CGI (Common Gateway Interface) form. The 27 questions had to be answered 
on a scale of 1 to 5 (lowest to highest). After each of these cycles, the students’ responses were summarized and 
graphically displayed within the site [Collis, Verhagen & Gervedink Nijhuis 1996b]. In the following 
paragraph some main results are mentioned from these on-going student evaluations of the 1996-97 cycle of 
the course. It is also indicated how the 1997-98 version of the course was redesigned, based on these main 
results. 

With regard to the results, the students were consistently pleased with the possibilities of the WWW- 
based course environment. Students expressed they liked working on the course through the ISM-1 site very 
much. They also liked the specialist tasks assigned to each person in the group-project part of the course, and 
were able to work on their tasks without help. The way lecture notes were available on-line before and after the 
classes was considered a good idea compared to other courses, where the notes were not available or only after 
class. 

Students also stated specific suggestions concerning future improvement of the course and the course site: 

• It is difficult to read from the screen, especially after some time. 

• Hyperlinked study texts take up too much time, especially when studied at home through a modem. 

• It is very easy to get lost because the information wanted is stored very deeply in the site. 

• The exercises are time-consuming, just doing projects is preferred. 

• The communication between group members has to be improved. 

While these points relate to the organization of the course and the course-site as a whole, points such as 
the following relate specifically to the user interface: 

• The navigation could be clearer. 

• The text is difficult to read. 

• It is not immediately clear where new things in the site are put. 
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• Consistency on the pages is required for better clarity. 

• It has to be immediately clear whether the current page is an internal or external page. 

• Italics are very difficult to read from a screen. 

• Tables sometimes do not fit on the screen. 

Specific design-related conclusions are inferred from these suggestions and also by use of traces of student 

usage of the course-site and other student comments made during the course: 

1. Students do not like to study by reading from the screen. They prefer printed, portable, linear study 
materials instead of hyperlinked interactive study materials so they can study them away from the 
computer. 

2. Students do not want to go further than three clicks (or levels) to find what they need. 

3. Students appreciate a navigation frame that is always present. They need a clear understanding of what is 

meant by each option in the frame. 

4. Students are sensitive to the readability of text on the screen, to its layout, and to a consistent screen design, 
based on the way that text is formatted and spaced, with a restful contrast between background color and 
the colors chosen for text and links. Highlighting features such as italics are not appreciated because of 
their impact on readability. Consistent and restful are key attributes. 

5. Students are not particularly interested in images and logos on the pages of the course-site, but they do 
appreciate being able to visually distinguish the course pages from external pages that are called up; thus 
all course pages have to possess a common look. 

6. Students prefer to scroll through a page, rather than to use internal links to move around within the page, 

but they do appreciate a consistent link from the bottom of the page to the top of the page. 

7. Students want a direct indication of what is new in a site, and where to find it, as soon as they enter the 
course environment. 

3. Redesign of the User Interface for the 1997-98 Course Environment 



The course was redesigned for the 1997-98 year based on these and other student reactions to the user 
interface. In the next paragraph the main way the evaluative comments of the students influenced the course 
redesign are described. 

Navigation . In 1996-97 students said that the navigation frame appearing on the left of every screen had to 
be clearer. Thus the navigation frame has been changed from icon based to word based, because the students 
indicated that it was not quite clear what the icons represented. As it is difficult to find suitable icons for the 
different centres, the option of word-based navigation for the navigation frame has been chosen. Students also 
said that one of the results of using a frame was that it sometimes was not possible to go back or forward using 
the buttons in the browser. Therefore backward and forward arrows are now provided on the pages. A home 
button is also provided at the bottom of each page. 

Layout. Consistency in the pages was required for better clarity. Thus all pages are now built using the 
same template. The template contains the background color, text color, title color, menu, ISM-1 logo, table 
format, bullets and horizontal bars. It was indicated that it has to be immediately clear whether the current 
page is an internal or an external page. Therefore different colors and the new (smaller) logo are used. The 
logo represents the KOPIE model [Collis & Verhagen 1997] which is the leading conceptual thread for the 
course-site. The Educational Science and Technology house-style colors inspired the choice of the colors blue, 
red and yellow. 

Icons. In last year’s course students said that the icons were not clear. Now the icons are renewed. As it 
was very difficult to find representative icons, headers are placed next to icons smaller than those of the 
previous year, to clarify their meaning; consequently an icon’s function is reduced from a navigation item to a 
decorative recognition item. 

Tables. The tables had to be clearer, thus a new template for the tables was designed, including no fixed 
table widths. Instead table width is indicated by a percentage of the screen, so that in this way the tables always 
adapt to the screen size. 

Text clarity. Last year it was indicated that the black text on the white background was difficult to read, 
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and not pleasing to the eyes when larger portions of text were read. The improvement made is that the text 
color has been changed to dark purple, and the background color to light yellow. The choice of these colors 
was directed by comments such as: the background was too light, and the text was too dark. 

Text readability. In the 1996-97 course students said that the texts were presented in paragraphs that were 
too long. Paragraphs are now shorter, and subdivided by horizontal bars. The main items are stated at the top 
of each centre in a menu. By clicking on these items the text immediately scrolls down to the requested part of 
the text. At each break an arrow to the top is presented that allows the user to easily return to the menu. 

As an example of those adaptations, the user interface from the 1996-97 course [Collis, Verhagen & 
Gervedink Nijhuis 1996a] is shown in Figure 1, while the revised user interface from the 1997-98 course 
[Collis, Verhagen & Gervedink Nijhuis 1997a] based on the student comments is shown in Figure 2. 
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Figure 1: Example of the interface design of the ISM-1 course-site from the year 1996-97. 
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Figure 2: Example of the interface design of the ISM-1 course-site from the year 1997-98. 





4. Student Evaluation of the Redesign 



Continuing our process of on-going student evaluations of the user interface of the course-site, 
students in the 1997-98 cycle were surveyed during the fifth week of the course (October, 1997) with regard to 
the user interface of the course-site. This survey occurred with selected students via interviews and an 
evaluative CGI-form available within the course [Collis, Verhagen & Gervedink Nijhuis 1997b]. Students were 
not forced to respond, but 38 out of the 60 students did within a one-week period. The results were directly 
summarized after this one-week period and made available to the students via the course-site [Collis, Verhagen 
& Gervedink Nijhuis 1997c]. Table 1 gives some of the survey questions and a summary of the student 
responses. 



Selected questions and answer options: 


Percentage of students 
responding to each 
option (N=38) 


What do you think about the color of the background? 




very bad 




0 


bad 


(a bit pale, a bit bright) 


2.8 


no opinion 




5.6 


good 


(nice to look at, better than a white background) 


55.6 


very good 




36.1 


What do you think about the color of the text? 




very bad 




0 


bad 


(not enough contrast, a bit too bright) 


2.8 


no opinion 




13.9 


good 


(good color, good contrast) 


61.6 


very good 




22.2 


What do you think about the size of the text? 




very bad 




0 


bad 


(too small, too large, too thin) 


5.4 


no opinion 




21.6 


good 


(good size, exactly right size) 


43.2 


very good 




29.7 


What do you think about the navigation frame? 




very bad 


(I don’t like to scroll) 


11.4 


bad 


(not clear what information the words represent) 


11.4 


no opinion 




11.4 


good 


(fast way of navigating) 


40.0 


very good 


(very good, clear) 


25.7 



Table 1: Sample of items from student evaluation of the redesigned user interface of the 1997-98 ISM-1 
course-site (Full results at [Collis, Verhagen & Gervedink Nijhuis 1997c]) 



5. Conclusion 

From the survey results and student interviews, as well as other sources of data such as traces of 
student usage of the course-site, it is shown that most of the user interface redesign choices are well received by 
the students. The major problem of site structure and how this is presented to the students via the user interface 
remains. The conceptual decision to present the course-site based on the metaphor of ‘centres’ (Information 
Centre, Study Centre, Production Centre, Specialist Centre, Communication Centre and Search Centre) was 
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taken as a basis for navigation, to organize the types of materials and interactions supported in each of these 
parts of the course-site. This decision is conceptually logical, and fits the instructors' mental models of the 
course-site. Students however, live and work in sequential time: finding what they need for a given week in as 
efficient a manner as possible may be a more appropriate approach for the organization of the site as expressed 
in its user interface. This is a serious point of consideration for further cycles of the course, and for the design 
of WWW-based course environments in general. 



6. Discussion 

Designing the user interface of a WWW-based course environment is a complex task, involving the 
combination of form, function, style and practicality. It is important that student evaluations of the user 
interface of these environments occur whenever they are used. Some of the design decisions that the course-site 
designers may feel are most appropriate may not be perceived by the students as most helpful. Examples of this 
are: organizing the course-site conceptually rather than by a week-to-week basis, offering the students 
hyperlinked study materials, and offering frequent internal navigation options within pages. Other aspects, 
such as the way text is laid out on the page and an overall peaceful feeling resulting from the colors and style 
of the site, may be more important than previously considered. Asking the students systematically and 
regularly about their opinions should become part of our standard practice when WWW-based course 
environments are used. This can be realized through a combination of CGI-form surveys, informal interviews, 
and examination of the logs of their movements within the site. 
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